Design and Evaluation of a SLDS for E-Mail Access through the Telephone

directed to make e-mail universally and seamlessly accessible to a broad population of potential users through an affordable telephone-based service. Thus, the main objective of E-MATTER was to develop a Spoken Language Dialogue System (SLDS) for an e-mail access service that uses a multilingual spoken language interface (both input and output) and that takes into account the cultural and the linguistic diversity nature of the e-mail messages. This paper addresses the different linguistic resources involved in the design and evaluation of the E-MATTER prototype. In the first part of the paper, we describe the guidelines for the design of the principal linguistic technologies involved in the development of the system: multilingual speech recognition, multilingual text-to-speech conversion, semantic parsing, dialogue management, language identification and advanced text verification. Then we present an evaluation methodology we have followed to obtain a complete analysis of both module deficiencies and global system behaviour. This methodology has been used to show us how to improve the prototype system, and we hope it will be general enough to be useful for testing other similar SLDS's. E-MATTER is a multilingual Spoken Language Dialogue System (SLDS) that provides e-mail access over the telephone by combining different technologies such as continuous speech recognition, text-to-speech conversion, semantic parsing, dialogue management, language identification and text verification (E-MATTER, 1999). The user is able to access his e-mail inbox using natural language over the telephone, select one of the incoming messages, listen to it, get it replayed and reply it with voice mail. The system's users are expected to be rather varied so the user profile should not be restricted to a specific user type but allow the user to configure his profile (native language, e-mail filters, etc). This is done through a windows-based Internet interface connected with a user database. The system can read e-mail messages in all the languages of Spain (Spanish, Catalan, Galician and Basque) and in two more European languages (English and French) and the system architecture has been designed to easily allow the addition of new languages. Users can interact with the system both in Spanish and Catalan. The dialogue follows a menu-like structure so that it is clear to the user how that structure is built and how to move up and down in the structure. Although it is similar to a menu structure, it does not have the restrictions that such a structure has, but it makes …